Goto

Collaborating Authors

 mission scenario


CoFineLLM: Conformal Finetuning of LLMs for Language-Instructed Robot Planning

Wang, Jun, Vorobeychik, Yevgeniy, Kantaros, Yiannis

arXiv.org Artificial Intelligence

Large Language Models (LLMs) have recently emerged as planners for language-instructed agents, generating sequences of actions to accomplish natural language tasks. However, their reliability remains a challenge, especially in long-horizon tasks, since they often produce overconfident yet wrong outputs. Conformal Prediction (CP) has been leveraged to address this issue by wrapping LLM outputs into prediction sets that contain the correct action with a user-defined confidence. When the prediction set is a singleton, the planner executes that action; otherwise, it requests help from a user. This has led to LLM-based planners that can ensure plan correctness with a user-defined probability. However, as LLMs are trained in an uncertainty-agnostic manner, without awareness of prediction sets, they tend to produce unnecessarily large sets, particularly at higher confidence levels, resulting in frequent human interventions limiting autonomous deployment. To address this, we introduce CoFineLLM (Conformal Finetuning for LLMs), the first CP-aware fine-tuning framework for LLM-based planners that explicitly reduces prediction-set size and, in turn, the need for user interventions. We evaluate our approach on multiple language-instructed robot planning problems and show consistent improvements over uncertainty-aware and uncertainty-agnostic finetuning baselines in terms of prediction-set size, and help rates. Finally, we demonstrate robustness of our method to out-of-distribution scenarios in hardware experiments.


Correct-by-Construction Design of Contextual Robotic Missions Using Contracts

Mallozzi, Piergiuseppe, Nuzzo, Pierluigi, Piterman, Nir, Schneider, Gerardo, Pelliccione, Patrizio

arXiv.org Artificial Intelligence

Effectively specifying and implementing robotic missions poses a set of challenges to software engineering for robotic systems. These challenges stem from the need to formalize and execute a robot's high-level tasks while considering various application scenarios and conditions, also known as contexts, in real-world operational environments. Writing correct mission specifications that explicitly account for multiple contexts can be tedious and error-prone. Furthermore, as the number of contexts, and consequently the complexity of the specification, increases, generating a correct-by-construction implementation (e.g., by using synthesis methods) can become intractable. A viable approach to address these issues is to decompose the mission specification into smaller, manageable sub-missions, with each sub-mission tailored to a specific context. Nevertheless, this compositional approach introduces its own set of challenges in ensuring the overall mission's correctness. In this paper, we propose a novel compositional framework for specifying and implementing contextual robotic missions using assume-guarantee contracts. The mission specification is structured in a hierarchical and modular fashion, allowing for each sub-mission to be synthesized as an independent robot controller. We address the problem of dynamically switching between sub-mission controllers while ensuring correctness under predefined conditions.


Optimizing Fuel-Constrained UAV-UGV Routes for Large Scale Coverage: Bilevel Planning in Heterogeneous Multi-Agent Systems

Mondal, Md Safwan, Ramasamy, Subramanian, Bhounsule, Pranav

arXiv.org Artificial Intelligence

Fast moving unmanned aerial vehicles (UAVs) are well suited for aerial surveillance, but are limited by their battery capacity. To increase their endurance UAVs can be refueled on slow moving unmanned ground vehicles (UGVs). The cooperative routing of UAV-UGV multi-agent system to survey vast regions within their speed and fuel constraints is a computationally challenging problem, but can be simplified with heuristics. Here we present multiple heuristics to enable feasible and sufficiently optimal solutions to the problem. Using the UAV fuel limits and the minimum set cover algorithm, the UGV refueling stops are determined. These refueling stops enable the allocation of mission points to the UAV and UGV. A standard traveling salesman formulation and a vehicle routing formulation with time windows, dropped visits, and capacity constraints is used to solve for the UGV and UAV route, respectively. Experimental validation on a small-scale testbed (http://tiny.cc/8or8vz) underscores the effectiveness of our multi-agent approach.


Automated Multidisciplinary Design and Control of Hopping Robots for Exploration of Extreme Environments on the Moon and Mars

Kalita, Himangshu, Thangavelautham, Jekan

arXiv.org Artificial Intelligence

The next frontier in solar system exploration will be missions targeting extreme and rugged environments such as caves, canyons, cliffs and crater rims of the Moon, Mars and icy moons. These environments are time capsules into early formation of the solar system and will provide vital clues of how our early solar system gave way to the current planets and moons. These sites will also provide vital clues to the past and present habitability of these environments. Current landers and rovers are unable to access these areas of high interest due to limitations in precision landing techniques, need for large and sophisticated science instruments and a mission assurance and operations culture where risks are minimized at all costs. Our past work has shown the advantages of using multiple spherical hopping robots called SphereX for exploring these extreme environments. Our previous work was based on performing exploration with a human-designed baseline design of a SphereX robot. However, the design of SphereX is a complex task that involves a large number of design variables and multiple engineering disciplines. In this work we propose to use Automated Multidisciplinary Design and Control Optimization (AMDCO) techniques to find near optimal design solutions in terms of mass, volume, power, and control for SphereX for different mission scenarios.


PUFFER: JPL's Pop-Up Exploring Robot

IEEE Spectrum Robotics

It costs a stupendous amount of money to send something from the surface of Earth to the surface of Mars, and there are severe limits on the volume and mass that you can send at any one time. In order to stuff the maximum amount of science into the minimum amount of space, NASA has had to get creative, with landers and rovers designed to be lightweight and foldable. At NASA's Jet Propulsion Laboratory, in Pasadena, Calif., engineers have long been trying to cram as much robot as possible into the absolute minimum amount of space, and a team of roboticists there recently showed us their latest creation: PUFFER, the Pop-Up Flat Folding Explorer Robot. It's designed to pack down nearly flat for transport, and then re-expand on site to investigate all the places a bigger rover can't quite reach. The overall idea with PUFFER is that you'd pack a bunch of them along with the next Mars rover, and send them out whenever you want to go somewhere that it would be either risky or impossible for the larger rover to go.


Real-Time Optimal Selection of Multirobot Coalition Formation Algorithms Using Conceptual Clustering

Sen, Sayan Dev (Vanderbilt University) | Adams, Julie Ann (Vanderbilt University)

AAAI Conferences

The presented framework is the The multirobot coalition formation problem seeks to intelligently first to leverage a conceptual clustering technique to partition partition a team of heterogeneous robots into any set of coalition formation algorithms in order to derive coalitions for a set of real-world tasks. Besides being N Pan optimal hierarchy classification tree, given any classification complete (Sandholm et al. 1999), the problem is also hard taxonomy. The results contribute to the state-ofthe-art to approximate (Service and Adams 2011a). Traditional approaches in multiagent systems by demonstrating the existence to solving the problem include a number of greedy of crucial patterns and intricate relationships among existing algorithms (Shehory and Kraus 1998; Vig and Adams coalition algorithms.